Information Entropy in Cosmology* 
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The effective evolution of an inhomogeneous cosmological model may be described in terms of 
spatially averaged variables. We point out that in this context, quite naturally, a measure arises 
which is identical to a fluid model of the 'KuUback-Leibler Relative Information Entropy', expressing 
the distinguishability of the local inhomogeneous mass density field from its spatial average on 
arbitrary compact domains. We discuss the time-evolution of 'effective information' and explore 
some implications. We conjecture that the information content of the Universe - measured by 
Relative Information Entropy of a cosmological model containing dust matter - is increasing. 
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A MEASURE OF INHOMOGENEITY IN THE 
UNIVERSE 

Cosmology is based on the hypothesis of simphcity 
called the cosmological principle, i.e. homogeneity and 
isotropy. The departure of the actual mass distribution 
from the homogeneous universe model is quantified in 
terms of density contrast or a statistical quantity like 
the two-point correlation function, which both have been 
studied either by perturbation theory or numerical sim- 
ulations. Behind these investigations there is a belief 
that the Universe is homogeneous on some large enough 
scale. This belief has to be quantitatively confronted 
with observation, explicitly introducing a measure of in- 
homogeneity for a domain of the Universe. 

In this Letter we propose a measure which quantifies 
the distinguishability of the actual mass distribution from 
its spatial average, borrowing a well-known concept in 
standard information theory. Suppose we are told that 
the probability distribution is {qi} and would like to ex- 
amine how close this distribution is to the actual one {pi\ 
by carrying out observations or coin tossing; the relevant 
quantity in information theory is the relative entropy, 



Pi 



(1) 



which is positive for qi ^ Pi, and zero if the actual dis- 
tribution {pi\ agrees with the presumed one {qi}. Note 



that this relative entropy is not symmetric for the two 
distributions {pi\ and {qi}. It is known that this mea- 
sure always decreases or stays the same under Markovian 
stochastic processes (i.e., a Zmear positive map). Namely, 
the actual distribution becomes less and less distinguish- 
able from the priorly informed distribution due to the 
random process. In cosmology we are interested in how 
the real matter distribution is different from its spatial 
average. For a continuum the relevant quantity would be 
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where q is the actual distribution and (• • ■)x> its spatial 
average in the volume on the compact domain 2? of 
the manifold E. We shall conjecture that the measure 
S{q\ I (^?)s} continues to grow indefinitely, if S is compact. 

The resolution of the apparent discrepancy between 
the gravitational system and the ordinary stochastic sys- 
tem will be, (i) we are considering in cosmology a non- 
isolated system defined by a comoving region T) in con- 
trast to an isolated system for an ordinary stochastic pro- 
cess, and (ii) the time evolution dictated by Einstein's 
equations induces a negative feed-back due to the at- 
tractive nature of the gravitational force, which tends to 
make the matter distribution more and more inhomoge- 
neous. 



DEDUCTION OF THE MEASURE 
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To begin with let us emphasize that the functional ||2J), 
known as the 'Kullback-Leibler Relative Information En- 
tropy' (c/ 0, 0, 0) is not assumed as a measure a 
priori, rather it can be deduced from a fundamental kine- 
matical relation that refers to the non-commutativity of 
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two operations: spatially averaging and evolving the ma- 
terial mass density field. The specific form of the infor- 
mation measure is, thus, inherently determined by the 
physical problem at hand, and does not need to be justi- 
fied empirically or axiomatically as is the common status 
of information measures in the literature. 

We define the averaging operation in terms of Rieman- 
nian volume integration, restricting attention to scalar 
functions 

{^it,X^))T,:^r^ [ ^d'X ^{t,X') , (3) 
Jv 

with the Riemannian volume element d^g := ^fgd^X, 
g := det((7ij), and the volume of an arbitrary compact 
domain, Vx)(t) := J^^y^d'^X; X^ are coordinates in a 
t = const, hypersurface (with 3— metric gij) that are co- 
moving with fluid elements of dust: 

ds^ ^ -dt^ + g.jdX'dX^ . (4) 

It is evident from the above setting that we predefine 
a simple time-orthogonal foliation (which restricts the 
matter to an irrotational dust continuum) in order to 
simplify the framework in which we discuss our measure 
as a concept of a spatial average. We wish to emphasize 
that the formalism below could be carried over to more 
general settings (e.g. to perfect fluids or scalar fields (c/. 
yf) with possibly further interesting implications. 

The above-mentioned 'non-commutativity' has been 
fruitfully exploited in previous work on the averaging 
problem of inhomogeneous cosmologies Q, |^, IE ^ 
and can be compactly written in terms of a commutation 
rule for the averaging of a scalar field ^f: 

= {^se}T> = {0S'ii)v = {S'^se)v , (5) 

where 9 denotes the local expansion rate (as minus the 
trace of the extrinsic curvature of the hypersurfaces t = 
const.). We have rewritten the r.h.s. of the first equality 
in terms of the deviations of the local fields from their 
spatial averages, := - {^)v and 59 := 9 - {9)t>. 

The key-statement of the commutation rule (0) is 
that the operations spatial averaging and time evolution 
do not commute. In cosmology we may think of ini- 
tial conditions at the epoch of last scattering, when the 
fluctuations imprinted on the Cosmic Microwave Back- 
ground are considered to be averaged-out on a rest- 
frame of a standard Friedmann-Lemaitre-Robertson- 
Walker (FLRW) cosmology. In this picture the evo- 
lution of the Universe is described by first averaging- 
out (or ignoring) inhomogeneities and then evolving the 
average distribution by a homogeneous (in the above 
case homogeneous-isotropic) universe model. A realistic 
model would first evolve the inhomogeneous fields and. 



at the present epoch, the resulting fields would have to 
be evaluated by spatial averaging to obtain the final val- 
ues of, e.g., the averaged density field. In particular, this 
comment applies to all cosmological parameters (see, e.g., 
and Q). Let us illustrate this statement for the mass 
density field. Setting * = £», Eq. © reads: 

{g)v + {9)v{g)v = {g + 9g)v ■ (6) 

Since the r.h.s. vanishes due to the continuity equation, 
we also have a continuity equation for the averages: 

{g)v + {9)v{g)v = , (7) 

which simply expresses the conservation of the total ma- 
terial mass, M-p = /p ^/gd^X g, in our comoving and 
synchronous gauge setting. 

A fairly general insight that, in principle, will not de- 
pend on some specialized setting, can be obtained by 
rewriting Eq. the notion of 'non-commutativity' 

mentioned above comes into the fore by observing that 
the time-evolution of the average density does not coin- 
cide with the average of the locally evolved density: 

{g)v-{g)v^{g9)v-{g)v{9)v^{Sg59)v ■ (8) 

For the fluctuation terms on the r.h.s., which would van- 
ish in the FLRW model without any perturbation, we 
can give a deeper interpretation. For this end let us ask, 
which functional will reproduce these terms upon per- 
forming the time-derivative. First, note that for the av- 
eraged expansion rate (0)u the corresponding functional 
is the volume according to 

{(^)v = ^=--iHj, . (9) 

The latter equality demonstrates that this quantity may 
be regarded as an effective Hubble function^ which will 
show up in our discussion later. 

Interestingly, the answer is provided, for gi > 0, by the 
functional 5{£i||(gi)p}, Eq. ||2Jl, so that the source of non- 
commutativity in Eq. © is given (up to the sign) by the 
production of Relative Information Entropy, defined as 
to measure the deviations from the average mass density 
due to the development of inhomogeneities: 

{g)v - {g)v = ■ (10) 

This measure can actually be inferred from its definition 
in phase space in terms of the one-particle distribution 
function for dust matter, i.e. the matter density multi- 
plied by a delta-function distribution in velocity space 
y. It is here, where generalizations of the matter model, 
e.g. supported by pressure, vorticity and/or velocity dis- 
persion could be implemented, resulting in more general 
entropies after taking velocity moments in phase space. 
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The reader may ask, whether this measure is superior 
to the density fluctuation measure, which also provides a 
generally growing and positive-definite valuation of the 
density distribution. Let us give some answers to this 
question before we proceed. 

A standard index of inhomogeneity in cosmology is the 
density contrast S := -^^^ and the derived positive mea- 
sure (Ap)^ := {q^)v — {q)v- The Relative Information 
Entropy or the distinguishability, Eq. jSJ, may have fur- 
ther implications by exploiting results from information 
theory. At the present stage we do not claim that this 
measure is superior to the density fluctuation, but rather 
it is complementary. This can be illustrated by pointing 
out that both measures are "cousins" in a 1-parameter 
family of inhomogeneity measures defined by 



J^a{g\\{g)v} 



■D 



a+1 



with a being a real parameter. In the limit a — > 
the formula reproduces the relative entropy, Ta^Q — > 
iS/Vp, whereas a = 1 reproduces the density fluctua- 
tion, Ta=\ = (A£i)^/(£i)x>. This interpolating formula 
is known as the Tsallis relative entropy. It should be 
emphasized that the limit a — s- is singled out as the 
only measure that exactly provides the source of non- 
commutativity with regard to the density evolution. 



PROPERTIES OF THE MEASURE 

The measure | (gi)©} forms one of the central con- 
cepts in information theory 0|; 5 = ("zero struc- 
ture" ) is attained by the homogeneous mass distribution, 
Q {q)v- 

First, for strictly positive mass density, g > 0, 
is positive definite, which can be readily con- 
firmed, i.e. it is indeed a measure. 

Let us have a closer look at the total time-derivative of 
our measure. Following from what has been said above, 
we may write the total Relative Information Entropy pro- 
duction as follows: 



S{g\\{g)v} 
Vv 



= -{dge)v = -{gS9)v = -{SgS9)v ■ (12) 



The last quantity is bounded according to Schwarz' in- 
equality, so that we obtain: 



S{g\\{g)v} 



= \{Sgd0)v\ < AgA0 , (13) 
with the positive-definite fluctuation amplitudes 

Ag := VWgF) ; := . (14) 



This inequality states that the temporal change of the ra- 
tio between the distinguishability of the density distribu- 
tion from the homogeneous distribution and the volume 
is bounded by the density and expansion fluctuation am- 
plitudes. We may say that the production of information 
in the Universe and its volume expansion are competing. 

We may look more closely at bounds as well as kine- 
matical and dynamical conditions for the total second 
time-derivative of | In we give sufficient 
conditions for the time-convexity of our measure. Let 
us put one of them into perspective. We consider the 
question under which condition the time-derivative of 
the Relative Information Entropy production is positive. 
A straightforward calculation provides: 



S_ 

Vv 



^-{Sg6e)v + {g)v{Aef 



(11) Raychaudhuri's equation. 



d ^ A-AnGg- 



6^ - 2a^ 



(15) 



(16) 



with the rate of shear a := \ ha'^.a-'i, the shear tensor 



aij being minus the trace-free part of the extrinsic cur- 
vature), together with the commutation rule (O yields: 

= 47rG(A£))2 + {g)v{Aef + U5g6e'')v + '2{6g5o^)v 

Vv 

> AnGiAgf - Ag^Ae^ + 2Aa^] + {g)v{A0)^ . 



The r.h.s. is positive, if 

1 A(lg^) + A(2a2) 

2 AO 



< V'^T^G(g), 



tp-D 



(17) 



where tp^ denotes the effective free-fall time on T). 

Eq. (|17|l provides a sufficient condition for the time- 
convexity of the Relative Information Entropy, which can 
be met, if gravity dominates over expansion and shear 
fluctuations. Time-convexity implies that entropy pro- 
duction eventually becomes positive, i.e. the structure 
eventually surfaces and its rate of formation increases. 



DISCUSSION AND CONJECTURE 

Looking at Eq. H12|) we appreciate that the source, i.e., 
the averaged Relative Information Entropy production 
density, can be positive or negative. In cosmology, the 
processes of a relative accumulation of matter (cluster 
formation) and a relative dilution of matter (void for- 
mation) create structure compared with the average dis- 
tribution. Following from Eq. 112|) . information entropy 
is produced if, on average, there are overdense fluid ele- 
ments {5g > 0) which are contracting (0 < 0), or under- 
dense elements {5g < 0) which are expanding {9 > 0), 
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respectively. With regard to cosmological structure for- 
mation these two states are generically encountered in a 
self-gravitating system, i.e., for large enough times and 
looking at some regional scale, an asymmetry of states 
is created due to the coupling of the expansion rate to 
the rate of change of the density through the continuity 
equation. We know from a calculation of the measure in 
linear perturbation theory that the growing-mode solu- 
tion supports states with {Sg > , 9 < 0} (contracting 
clusters) and {6q < , 9 > 0} (expanding voids). Thus, 
for sufficiently large times, i.e. when the decaying mode 
disappears, our measure will increase. 

Looking at Eq. H17|l we conclude that also in the case 
of the second time-derivative we have the possibility 
of time-concavity of the Relative Information Entropy. 
However, we have evidence that, at least for large enough 
times and on sufficiently large scales, time-convexity al- 
ways holds for a self-gravitating continuum of dust. In 
particular, in the linear perturbation theory, our measure 
is always time-convex [3. 

We can illustrate roughly the physical content of the 
sufficient condition H17|) as follows. Concentrating on the 
linear regime by considering the case in which fluctua- 
tions of a quantity are small compared with their aver- 
age values, we may expand the quadratic expressions and 
keep only the leading terms: 

A{9^) « 2\(9)v\A9 ; A(a') « 2\(a)v\A<j . (18) 

In this limit, if we additionally think of a large domain 
featuring approximately vanishing average shear, (cr)i5 ~ 

0, the sufficient condition (|17l) reduces to the inequality 

tp^ < =: \H-'\ , (19) 

1. e., if the effective free-fall time on V is smaller than the 
effective Hubble time t//^, with H-p defined in Eq. lO, 
then time-convexity of our measure is ensured under the 
given assumptions. The expectation that both positiv- 
ity of the Relative Information Entropy production and 
time-convexity, which are supported by the linear per- 
turbation results, will hold in the dust continuum gener- 
ically, at least for large enough times and on sufficiently 
large scales of averaging, establishes the following. 

Conjecture: The Relative Information Entropy of a dust 
matter model is, for sufficiently large times, 

globally (i.e. averaged over the whole compact manifold 
S) an increasing function of time. 

We are currently investigating nonlinear exact solu- 
tions for spherically-symmetric domains 7], which may 
provide further support for our conjecture. 

A note is in order as for the relation to observational 
constraints. In our context a generalized form of Fried- 
mann's differential equation governs the averaged expan- 
sion 0, and a set of four effective cosmological param- 
eters can be defined 0, 0. Assuming that, on suffi- 
ciently large scales of averaging, kinematical fluctuations 



and the averaged 3— Ricci curvature have negligible con- 
tributions, then the sum of the cosmological parameters 
for the matter content and the cosmological term have to 
add up to 1; the former is indeed given by the fraction of 
the two competing times: 

„ _ 8TrG{g)v _ 2 tj^^ 

Refering to observational results, e.g. by WMAP 
(Wilkinson Microwave Anisotropy Pro&e')[loj. its contri- 
bution is « 0.3 and, thus, tpj, is slightly larger than 
tn-o ■ Note that this does not immediately imply that 
our measure is not time-convex, because the condition 
(|19|l derives from the sufficient condition (|17|l , which only 
provides a rough estimation and is not very stringent. 
On cosmological scales both times are indeed very simi- 
lar, so that we should make the estimation tighter to see 
whether or not time-convexity holds; this we postpone 
to the future work . 

We contemplate that the measure that we propose in 
the present Letter not only incorporates an assessment of 
structure, but may turn out to be a fundamental quantity 
in many other respects, e.g. for the study of Black Holes 
and the Early Universe. 

We would like to thank Alvaro Dominguez, Jose Gaite 
and Atsushi Taruya for constant interest and discussions. 
TB acknowledges hospitality during a visit in 2001 at 
the Hosoya Laboratory of the Tokyo Institute of Tech- 
nology, where the main work on this subject was done 
with support by The Japanese Society for the Promotion 
of Science (JSPS). He also acknowledges hospitality at 
the University of Tokyo with support by the Research 
Center for the Early Universe (RESCEU, Tokyo), COE 
Monkasho Grant, where this Letter was written. This 
work is also partially supported by the Sonderforschungs- 
bereich SFB 375 'Astroparticle physics' by the German 
science foundation DFG. 



[1] T. Buchert, G. R. G. 32, 105 (2000). 

[2] T. Buchert, G. R. G. 33, 1381 (2001). 

[3] T. Buchert, in: 9th JGRG Meeting, Hiroshima 1999, Y. 

Eriguchi et al. (eds.), pp. 306-321 (2000). 
[4] T. Buchert and M. Carfora, Phys. Rev. Lett. 90, 31101- 

1-4 (2003). 

[5] T. Buchert and J. Ehlers, Astron. Astrophys. 320, 1 
(1997). 

[6] T. M. Cover and J. A. Thomas, Elements of Information 

Theory Wiley, N.Y. (1991). 
[7] A. Hosoya, T. Buchert, and M. Morita, in prep. (2004). 
[8] S. Kullback, Information Theory and Statistics, Wiley, 

N.Y. (1959). 

[9] S. Kullback, R. A. Leibler, On information and suffi- 
ciency, Ann. Math. Statistics 22 (1951), pp 79-86. 
[10] D. N. Spergel et al. Ap.J. Suppl. 148, 175 (2003). 



